Self-sustained activity in a small-world network of excitable neurons 
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We study the dynamics of excitable integrate-and-fire neurons in a small-world network. At 
low densities p of directed random connections, a localized transient stimulus results in either self- 
sustained persistent activity or in a brief transient followed by failure. Averages over the quenched 
ensemble reveal that the probability of failure changes from to 1 over a narrow range in p; this 
failure transition can be described analytically through an extension of an existing mean-field result. 
Exceedingly long transients emerge at higher densities p; their activity patterns are disordered, in 
contrast to the mostly periodic persistent patterns observed at low p. The times at which such 
patterns die out are consistent with a stretched-exponential distribution, which depends sensitively 
on the propagation velocity of the excitation. 

PACS numbers: 05.45.-a,87.18.Sn,87.10.+c 



Recent research in complex networks has provided in- 
creasing evidence for their relevance to a variety of phys- 
ical, biological, and social phenomena [H 0- 0| • Two dis- 
tinct types of topology have been particularly useful in 
providing insights into the implications of complex con- 
nectivity: scale-free networks 3], characterized by the 
existence of a small number of hubs with high coordina- 
tion number, and small- world networks 0, characterized 
by the presence of shortcuts that link two randomly cho- 
sen sites regardless of the distance between them. 

So far, most work on complex networks has focused on 
their topological and geometrical properties; less atten- 
tion has been given to the properties of dynamical sys- 
tems defined on such networks. The interplay between 
the intrinsic dynamics of the constituent elements and 
their complex pattern of connectivity strongly affects the 
collective dynamics of the resulting system. For instance, 
the addition of shortcuts induces a finite-temperature 
hase transition even in the one-dimensional Ising model 
and the introduction of unidirectional shortcuts can 
change the second-order phase transition in the two- 
dimensional Ising model into a first-order one @. In a 
system of coupled oscillatory elements, the introduction 
of shortcuts enhances synchronization [||, while the in- 
troduction of hubs eliminates the threshold for epidemic 
propagation 0. 

The coexistence of shortcuts and regular local con- 
nections characteristic of small-world networks (SWNs) 
mimics a salient feature of the circuitry in the cortex 
H, IU 0, O, 0, El, where experimental observations 
of excitatory traveling waves provide evidence of 
some degree of local connectivity, while it is also recog- 
nized that long-range excitatory connections are present 
[ToL IT^I . Our goal is to explore the influence of this com- 
plex connectivity on the dynamics of neuronal circuits; 
to this purpose, we choose a minimal model. The un- 
derlying network is modeled as a SWN with unidirec- 
tional shortcuts that reflect the nonreciprocal character 
of synaptic connections, and the excitable neurons are 
modeled as leaky integrate-and-fire units. We find that 



even this simple model exhibits a rich repertoire of dis- 
tinct dynamical behaviors as a function of the density p of 
added shortcuts: a low p regime characterized by persis- 
tent periodic activity that is bistable with the quiescent 
state, a transition to failure with increasing p, followed 
by a reemergence of long-lasting disordered activity. We 
note that a SWN with unidirectional shortcuts has been 
considered in a different regime by Lago-Fernandez et al 
[f3. ITEj to address the possibility of rapid synchroniza- 
tion among conductance-based neurons of the Hodgkin- 
Huxley type. 

The model considered here consists of a one- 
dimensional array of N integrate-and-fire neurons (IFNs) 
in which a SWN topology is created through the addition 
of a density p of unidirectional long-range couplings. The 
membrane potential of the IFNs is determined by 
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A neuron fires whenever its voltage exceeds 1; the voltage 
is then reset to 0. The external current is chosen to satisfy 
I ex t < 1; in this regime the IFNs are not oscillatory, but 
excitable. The last term models input currents due to 
presynaptic firing as a delayed impulse: if Wij is non-zero, 
then neuron i receives a pulse input of amplitude g syn 
with a delay rrj after neuron j has fired its mth spike at 
time tj . The synaptic conductance is chosen to satisfy 
Iext + gsyn > 1 , so that a single input suffices to sustain 
firing activity. The local connections are modeled here 
as nearest-neighbor couplings (w^i±i = 1) that define an 
underlying regular lattice. The long-range connections 
result from randomly adding rather than rerouting a 
fixed fraction pN of unidirectional couplings Wij = 1 to 
generate a SWN topology. 

At p = 0, any excitation sufficient to cause a neuron 
to fire will generate two pulses that propagate through 
the regular lattice in opposite directions with velocity 
v = 1/td, and either exit the system or annihilate each 
other, depending on boundary conditions. No persistent 
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FIG. 1: Raster plot (top) and instantaneous firing rate (bot- 
tom) for a system with N = 1000, I ex t ~ 0.85, g ay n = 0.2, 
T m = 10, m = 1, p = 0.1. Same parameter values are used in 
subsequent figures unless noted otherwise. 
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FIG. 2: Inset: Failure rates for to = 1.0 and different system 
sizes N. Main: Failure rates normalized by p cr (N) fcf . 13141 ") . 



activity results in either case. However, self-sustained 
activity may arise for nonzero p, as shown in fig^ Per- 
sistence relies on the reinjection of activity via a shortcut 
into a previously active domain that has by then recov- 
ered; this reinjection can occur only if the shortcuts are 
unidirectional. For a fixed value of p, any particular net- 
work realization has a different connectivity graph that 
may or may not sustain persistent activity. We average 
over typically 2000 realizations to calculate the probabil- 
ity of persistent activity; the complementary probability 
of failure to sustain activity is shown in fig|21 (inset) as a 
function of the density p of random connections for four 
different system sizes. In this regime, the probability of 
failure is an increasing function of p that crosses over 
from to 1 with increasing steepness as the size N of the 
system increases. 

Failure to sustain activity is a simple consequence of 
the intrinsic dynamics of the neurons. Pulses travel out- 
wards from an initial activity seed, and spawn new pulses 
at a rate that increases with p. A currently inactive neu- 
ron can fire again only if enough time has elapsed from its 
preceding firing to allow for a recovery to V > 1 — g syn - 
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FIG. 3: Failure rates after t max = 2000 for t d = 0.6, 0.8, 1.0, 
1.2, 1.4, 1.6, 1.8 (left to right). 



A single input will be able to elicit a spike only if the 
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If activity recurs to a given site too rapidly, the neuron 
will fail to produce a spike, and the pulse of activity will 
die out. A critical density p cr for the transition from 
self-sustained activity to failure can be estimated from 



Ta(Pct) = T j 
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where Ta (p) is the time needed for the activity to spread 
across the whole network. At a fixed velocity for pulse 
propagation, this time corresponds to the largest distance 
across the network. This distance has been calculated for 
bidirectional shortcuts using a mean-field approach [lfij : 
when extended to the case of unidirectional shortcuts it 
results in 
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An implicit expression for p cr and its dependence on 
the system size N and the propagation velocity v fol- 
lows from combining equations © and Q; for large 
N, p cr {N) oc InN. Failure rate curves as a function 
of (p ~ p cr (N))/p cr (N), shown in fig|21 cross at the theo- 
retically predicted value. This observation, together with 
the increased steepness of these curves with increasing TV, 
indicates that a well-defined transition to failure occurs 
in the thermodynamic limit. 

This well-defined transition to failure occurs only for 
sufficiently fast waves, i.e. for short delay rjj. For larger 
tjj the dynamics of the system become quite more com- 
plex, and the fraction of realizations that fail before a 
fixed time (t max = 2000 in figlHJ) becomes a nonmono- 
tonic function of p. While at low p the firing patterns 
are highly regular (cf. fig^) and all failures occur within 
one or two cycles of the initial activity, for higher p the 
patterns are more disordered (cf. fig0J) and the activity 
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FIG. 4: Raster plot (top) and instantaneous firing rate (bot- 

(i) 



torn) of neurons with I SI > T^> (black) and I SI < T 
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(red) for td — 1.5 and p — 1.0. Dotted line is the total firing 
rate. 



FIG. 5: Failure-time distribution. Inset: Cumulative distri- 
bution of failure times at p = 1 for 1.5 < td < 1.7. Main: 
Failure rates at 5, 10, 20, 40 and 100 multiples of = 28.3 
(black to brown). Symbols are averages over 2000 realizations. 



can persist for a very long time before failure. Conse- 
quently, the distribution of failure times, shown in figGO 
exhibits an increasingly long tail for longer delay times. 

To understand the persistence of activity beyond p cr , 
it is important to recognize that the result for the crit- 
ical density p cr (N) hinges on the assumption that each 
neuron receives but a single excitatory input during each 
cycle of network activity. Its recovery time is therefore 
given by \ which sets a lower bound for the interspike 
interval (ISI) . While this assumption is well satisfied for 
small p, it does not hold for p = 0(1). In fact, the 
likelihood that a neuron has n incoming shortcuts fol- 
lows a multinomial distribution such that the fraction of 
neurons receiving two incoming connections grows from 
about 0.05% at p = 0.1 to about 30% at p = 1. Neu- 
rons that receive n inputs can have a recovery time as 
short as Fig0| reveals that such neurons, with 

ISIs lower than \ play a crucial role in maintaining 
network activity in the regime p ~ 1. While neurons 
with ISIs greater than J 1 ! (marked black in figBJ can 
go through silent epochs with near-zero activity, neurons 
with shorter ISIs (marked red in fig0J may fire two or 
three times during a network cycle and carry over the 
network activity across these silent epochs. 

The persistence of activity beyond p cr depends sen- 
sitively on td', this reflects the fact that persistence at 
higher densities p is due to chains or trees of neurons 
that bridge the silent epochs due to their multiple inputs. 
Since activity propagates with a fixed speed, a chain of 
multiple-input neurons of given length can bridge a time 
interval proportional to Trj. Thus, as td increases, ever 
shorter chains can contribute to bridging a silent epoch 
of given duration; the likelihood for failure will decrease 
accordingly. This picture is of course overly simplistic: 
whether a topological chain can be utilized as a dynamical 
bridge over a given silent epoch depends on the amount 
and timing of the inputs it receives, which in turn de- 
pend on the recent history of the entire network. Simu- 



lations reveal that the identity of the neurons that form 
the 'bridging' dynamical chains varies from cycle to cy- 
cle in an irregular way. This implies that the effective 
utilization of a dynamical chain over one cycle does not 
guarantee its availability on the next cycle. Therefore, 
even systems that persist for long times may still have a 
finite probability of failing. 

The cumulative failure distribution function F(t), 
shown in fig [5] for various values of td, exhibits a long 
tail and is well fit with stretched exponentials: F(t) = 
/oo (td) - Ce~ atf> with (3 w 0.4 (dotted lines). Even 
though the fits are based on runs up to t = 300, 000 for 
T£> = 1.65 and rrj = 1.7, they do not provide a value 
of /oo (td ) accurate enough for establishing whether true 
persistent activity exists for a small fraction of the net- 
work realizations (0.97 < Joo^d = 1.65) < 1). Strik- 
ingly, the dependence of F(t) on the delay time exhibits 
a high degree of structure, suggestive of 'resonances' at 
values of td for which some chains and trees can be op- 
timally utilized. 

One of the salient features of the emergent dynamics 
of the model is persistent self- sustained activity at low 
densities p of shortcuts. In this regime, the network is 
in fact bistable between 'off' and 'on' states, and can be 
switched between them with sufficiently large stimuli, as 
illustrated in figEI The synchronous stimulation of a suf- 
ficiently large number of neurons while the network is in 
the 'on' state increases the level of activity and effectively 
pushes the network to the right of the failure transition 
(cf. fig[2J), causing a transition into the 'off' state. 

To address the fact that real neuronal systems are 
noisy, the simulation shown in figE] includes Gaussian 
fluctuations in the membrane potential; their amplitude 
is chosen so as to cause the neurons to spike irregularly at 
a low rate. To keep the noisy spiking of a single neuron 
from generating a traveling pulse and initiating the 'on' 
state, we adjust the synaptic conductance so that several 
adjacent neurons must fire in rapid succession in order to 
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FIG. 6: Raster plot (top) and instantaneous firing rate (bot- 
tom) for p = 0.10 and k = 5. About 10 adjacent neurons 
are stimulated synchronously at t = 250; about 20% of the 
neurons are activated at t = 750. 



propagate a pulse of excitation. The network topology 
is modified accordingly: we extend the local coupling to 
include up to 2k neighbors (u>j,i±j = 1 for j = 1 . . . k) 
and model long-range connections via a population of in- 
termediate excitatory neurons which both receive input 
from and project to multiple adjacent neurons. Under 
these conditions, spontaneous activity is highly unlikely 
to initiate traveling pulses. However, a sufficiently large 
stimulus, synchronous across several neurons, can again 
turn the state of elevated activity on and off (cf. fig©; 
bistability is thus robust with respect to noise. 

Network bistability has been hypothesized to be the 
neural correlate underlying the type of short-term mem- 
ory known as working memory in the prefrontal cortex 
of monkeys and humans. Much more realistic and phys- 
iologically plausible models of cortical layers have been 
studied within the context of working memory (e.g. |17|). 
Yet, not much attention has been given to heterogeneities 
in network topology or to long-range excitatory connec- 
tions. The work presented here suggests that closer at- 



tention be given to the role of connectivity as an addi- 
tional factor that contributes to the generation of the 
persistent, active state associated with working memory. 

In conclusion, we have investigated the effect of in- 
corporating random unidirectional shortcuts to a one- 
dimensional network of locally coupled integrate-and-fire 
neurons. We find that even a very low density of short- 
cuts suffices to generate persistent activity from a local 
stimulus through the reinjection of activity into previ- 
ously excited domains. As the density of shortcuts is 
increased, the substantial decrease in the effective sys- 
tem size characteristic of small-world networks causes a 
crossover into a regime characterized by failure to sus- 
tain activity for essentially all network configurations. 
For sufficiently slow propagation velocities of the activ- 
ity and sufficiently high shortcut densities, an intriguing 
second crossover occurs into a regime in which the ac- 
tivity still fails but only after often exceedingly long and 
strongly chaotic transients. 

The complex dynamical phenomena we find in this 
extremely simple model are based on a robust mecha- 
nism: propagating pulses of activity that are sustained 
by branching and reinjection. We therefore expect that 
more realistic models of neuronal networks, which may 
include multiple ion channels and continuous synaptic 
currents as well as inhibitory coupling, will show qualita- 
tively the same behavior upon the addition of shortcuts 
if they originally support propagating pulses of activity 
that annihilate upon collision. Preliminary computa- 
tions show that the failure transition persists if shortcuts 
are not allowed to exceed a maximal length L m , as long 
as L m TD > Tjp 0]. The phenomena should also be ac- 
cessible in excitable chemical systems [2(|, in which the 
shortcuts could be implemented using video feedback. 
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